Method and apparatus for keeping a check on the storage time for goods in a storage

ABSTRACT

Method and an apparatus for registering and supervision of the positions and time of storage of articles entered into a cabinet or other delimited space. A preferred fields of application is refrigerators in which one or several cameras at selected occasions take pictures of the interior of the refrigerator. These pictures are treated in an image analyzer which transfers clear pictures of all articles stored on each shelf in the refrigerator to a computer for registration. In the computer there is stored for each article information on the time of entry in a list containing article denominations. For distance communication the computer can be connected to a central computer having information about all articles marketed on a national basis and including both pictures and denominations.

BACKGROUND OF THE INVENTION

The present invention refers to a method for recording the position ofgoods stored in a cabinet or other space, in particular food-stuffsstored in a refrigerator, and for supervision of the storage timethereof. For carrying out the method an apparatus, suitable therefore,has been invented.

DESCRIPTION OF THE RELATED ART

In SE-A-9900582-9 a cabinet is described which includes one or severalbuilt-in cameras for taking pictures of the content of the cabinet andtransferring the pictures to an image analyser. A number of picturestaken at different occasions are put together by the image analyserwhich by means of an associated computer makes lists of the content ofthe cabinet and, by means of a data program, of what to replenish. Ifthe cabinet contains food-stuffs no information on the age thereof or onwhich food-stuffs have to be consumed first can be obtained by the knownarrangement.

From practical use it has appeared that a refrigerator is loaded withfood-stuff items to such degree that not even with the aid of themirrors mentioned in the patent application referred to pictures areobtained by means of which all items will certainly be identified.Investigations have proved that in a household up to 10.000 articles canexist and surprisingly many of these are stored in a refrigerator.Equipment which offers a complete distance communication in terms of amobile phone connection from a store to a home computer in order toobtain information on articles to shop, of course, also has to include acabinet for groceries. Accordingly, an image analyser can be neededwhich has the capacity of a computer for identification of the 10.000articles indicated. In itself, this is a question of dimension and ofthe number of cameras to be used. However, the problem remains toidentify an article from fragmentary partial pictures only of saidarticle or its package for recognizable illustration, for examplethrough a picture or, alternatively, a denomination, transferred viaInternet to a shop computer.

Another problem to which, hitherto, the prior art has not offered anysolution is to automatically keep a check on the age of perishables, forexample stored in a refrigerator, in other words the “best-before-date”,actually the “usable-up-to-date”. In the known arrangement, in whichpictures are taken every time the refrigerator door is opened, or with adifferent selected frequency, one is reduced to using the conventionalmethod. As known, this involves a “best-before-date” printed onto afood-stuff package or manually written thereon. In connection withhome-made provisions or leftovers more frequently a making or packingdate is noted on an adhesive label. In both cases it is required thatthe article be removed from the storage cabinet for reading of the dateindicated thereon.

Inter alia fresh fruit and vegetables, offered for sale loose by theweight by self-service, do not naturally bear any indication ofrecommended time of consumption. In case of a desire of providing arefrigerator with an automatic warning device indicating the presencetherein of eatables no longer fit for consumption one is reduced to theuse of devices usually referred to as electronic noses. Such devicescannot sense smell in human meaning but detect the presence ofputrefactive gases, such as methane, hydrogen sulfide and amines, and inthe most developed forms also pheromones. However, these devices aredisqualified for domestic use because they are too expensive. Inaddition, the effect thereof comes too late when putrefaction hasalready started. Also, the devices cannot single out a specific articlefor which the date of consumption has expired.

SUMMARY OF THE INVENTION

The object of the invention is to bridge the weaknesses associated withthe present system as far as keeping a check on specific articles andthe storage time thereof in a space are concerned. The apparatusinvented for carrying out the new method is capable of identifyingarticles, recording the time of placement thereof in a storage space,such as a cabinet, and of indicating, in good time before the expirationof the time of consumption thereof, that the specific article should beconsumed in the immediate future. In case of eatables the arrangementcan also submit recipes in which the specific eatable, in the amountavailable, is contained.

The characteristic features required for carrying out the new methodappears from the appending claims as do those for the inventedapparatus.

For carrying out the method the apparatus includes the followingcomponents:

-   cameras, suitably with a picture size and in a number so as to cover    every partial space of the room or cabinet to be supervised-   a local computer with a program for recognition of articles    reproduced on pictures transferred by the cameras to an image    analyser, the computer in a practical case, from a functional point    of view, being split into separate units;-   preferably, also communication means for connection with a central    computer in which identities for all articles offered for sale on a    national basis are recorded, said central computer in certain cases    being adapted only to transfer the picture whereas the analysis is    performed by the central computer.

The local computer, i.e. the computer provided adjacent the room whichis supervised, the cabinet or cabinets can be connected to a keyboardand to a monitor or to a voice-controlled communication system. Incertain cases a mobile phone or an Internet-connected equipment can beused. In a suitable place in the cabinet or room an area or a shelf isprovided in which when introduced into the space articles can be placedfor identification by means of a bar code reader or a RFID-reader or forrecording of new articles by means of voice communication. Thisequipment is intended to be used in case neither the local computer northe central computer, consulted via an Internet connection, canrecognize an article. Such a case may occur if the article has beenpurchased during a trip abroad or if it has been imported. If thearticle is packed and provided with a bar code the central computershould be able to identify the article and if it fails the user has toenter the denomination of the article in the local computer by means ofthe keyboard or by means of the voice.

Alternatively, the article is given a running number and a denominationwhich both are recorded together with a picture of the article taken bya camera. The time of recording is stored. In case of an article withlimited tenability the “best-before-date” is also recorded in thecomputer or taken from a register for the corresponding type of goods.

In case of RFID-marking a corresponding recording method has to be used.

The invention can be applied in connection with storage spaces of allkinds from wholesale dealer stores to household medicine cabinets. Inthe first-mentioned case the invention can serve as a means for maximumdegree of packing as pallets with a specific article do not require adetermined place in a stand but can be placed anywhere there is an emptycompartment. Accordingly, the same type of articles can appear inseveral different places in the store; the computer keeps a check onwhere and on which pallet loads in the store are the oldest.

Another area of application can arise when shops in sparsely built-upareas shall serve also as a post office replacement. In most cases theadditional business will be required to be housed in the space alreadyprovided. As far as the post packages are concerned they have to bestored in a very compact way from arrival to delivery. The possibilityof using every square decimeter of the storage shelves provided willbecome even more a necessity with increasing e-shopping and mail-orderbusiness.

The major use of the invention, as counted in separate installations, isassumed to apply to private households, in particular relating torefrigerators and freezers, however not excluding refrigerating andfreezing rooms in connection with professional kitchens.

Below, the method will be described in connection with the use of theapparatus for a domestic refrigerator. This shall be taken as an exampleonly which should not be interpreted as a limitation of the inventiveidea whatsoever.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference is made to the appending drawing, in which:

FIG. 1 illustrates the schematic component assembly of the apparatus;

FIG. 2 shows a refrigerator equipped for carrying out the method;

FIG. 3 shows an enlarged view of a shelf shown in FIG. 2 and used forseparate identification of articles, and

FIG. 4 schematically shows the principle of finding and identifying ahidden article.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A refrigerator 1 having one or several doors 2 is provided with one orseveral cameras 3. The number of cameras is chosen in dependence on thenumber of partial spaces in the refrigerator, i.e. the number of shelves4 that are provided. The cameras are provided with wide angle lenses andare adapted to take digital colour pictures which are transferred to alocal computer 5 connected to an image analyser 6, said transfer takingplace via a fixed line or via radio link, for example by use ofso-called bluetooth technique. Suitably, the computer 5 is equipped witha keyboard 7 and/or means for voice communication, and a monitor 8. Thecomputer can also be connected to a central computer 9 via a fixed line,Internet or a radio link. Preferably, the central computer can belocated at the company that provides the apparatus (white goods dealeror a company offering E-shopping). It should be mentioned that the localcomputer does not need to be a physically separate unit but could bepart of the central computer.

In the central computer 9 all article identities are stored, ifapplicable recorded by means of bar codes or information stored onmagnetic strips of RFID-type. Vegetables and fruit, for example, whichappear unpacked in the store, are stored in the form of colour picturesas are articles without identifying characteristics as referred toabove. All packed articles are supplementary registered as colourpictures for reasons given below.

In performing the method the following operations are applicable.Articles from the shop are put in the refrigerator 1 and the door 2 isclosed. By means of a delay function the refrigerator light remains onduring a minute or so and during that time pictures are taken by thecameras of the articles on all shelves.

In an apparatus having cameras 3 mounted in the door 2, alternatively,the cameras take one or several pictures at different angles during theturning movement of the door in the angular area from 90° to a positionwhere the door is closed. On the monitor 8 the computer 5 canautomatically show pictures of the articles put in the refrigerator,shelf by shelf or the user must effect this stepping of the pictures bymeans of the computer keyboard 7 or by means of voice control. As far asin the computer 5 pictures have been recorded for all articles put intothe refrigerator 1 at this occasion nothing happens. However, in casethe computer should not recognize certain articles these will be shownon the monitor 8, for example marked in a differing colour or strongerilluminated. In that case the user can connect the local computer 5 withthe central computer 9 ans ask for the article identities. On theassumption that the central computer 9 can identify the articles theiridentities are transferred to the local computer 5 and recorded thereinfor future use, both as a picture and a denomination.

In case the central computer 9 does not recognize an article put intothe refrigerator 1 this article will be shown on the monitor 8 with asurrounding frame or in another way clearly marked, for example in alist of articles. This is an invitation to the user to put the articleconcerned in an area specifically marked or on a particular shelf 10provided in the refrigerator. Adjacent the shelf 10 one or severalmirrors 11 are provided which reproduce also the back and bottom sidesof the article in the pictures taken by the cameras 3. The article isdepicted by at least two cameras 3. If the article is recognized by thelocal computer 3 or the central computer 9 when more than one projectionthereof is available the user can press the ENTER-key 7 on the keyboardand registration in the local computer 5 is completed. In case none ofthe computers recognizes the article the user must enter thedenomination of the article by means of the keyboard 7 or by means ofvoice control and the denomination is then recorded in the localcomputer together with pictures of the article.

Preferably, the shelf 10 is made of transparent material, such as clearglass. In refrigerators having glass shelves or bar shelves noparticular shelf 10 is required but a marked area on a regular shelf canbe used for the purpose indicated. In a freezer, however, having shelvesformed by cooling elements as well as in cabinets for groceries withshelves made from solid wood or of fibre material a shelf 10 oftransparent material is required. The reason for this is that one camera3 depicts an article from above and another one from below through theshelf 10. Another advantage is that in the shelf 10 a load sensor can beprovided which is connected to the two cameras taking pictures causingthem to readjust focus so that sharp pictures are obtained of the newarticle. Advantageously, each shelf can be provided with a load sensor.These sensors and associated pictures of the shelf concerned canindicate if an additional article has been added or, in case of reducedweight and the image memory of the computer does not indicate any changeof the number of articles or their identity, if in a package a certainquantity has been removed. If the computer can determine that anypackage has been disarranged the computer records that part of itscontent has been consumed.

In use of the method in a warehouse environment it is assumed thatpallets or cardboard boxes or the like piled on pallets are marked bymeans of bar code labels. Each pallet position is equipped with a loadsensor which upon a change (increase or decrease) of the load activatesa camera, suitably situated on an opposite stand. The latest picturetaken is stored in the computer and, accordingly, the computer cansignal when the ordering point for the article concerned has beenreached.

In the case of a post office the method is carried out in a similar way.Arriving parcels are placed in any free position in the available shelfcompartments. Then, pictures are taken of the shelves by a sufficientnumber of cameras. The identities of the parcels are represented by barcode labels turned outwards. On the parcel dispatch note an identicalbar code has been printed. Upon collection the latter bar code is readby a bar code reader connected to a local computer. The computerprovides an indication of a shelf compartment making possible for theserving officer to find the parcel in question. If for some reason theparcel has arrived in the wrong collecting place, by means of the localcomputer a question can be directed to a central computer for the placeof this parcel. In that way, by means of the local computer the parcelcan be ordered from the erroneous collecting place.

In order to facilitate the supervision by the apparatus of for examplethe degree of freshness of food in a refrigerator a virtual system ofcoordinates is provided in the local computer. This means that theposition of an article in the refrigerator is known by the computer 5.As regards packed food not completely consumed at one occasion butreturned into the refrigerator for example after a meal, the computercan react in any of a few different ways. If the food is packed in atransparent vessel, e.g. sliced cucumber in a jar, the computer 5 cannote that the volume of the food in the jar has decreased. In case ofbeverages packed in a non-transparent package (e.g. milk in carton pack)the computer assumes that a certain volume has been consumed. This isshown on the monitor 8 by giving the package a different colour or byindicating in the content list the number of times the package has leftthe refrigerator.

Due to the fact that changes of the content of the cabinet 1 can onlytake place when the cabinet door is open it is sufficient that picturesare taken by the cameras 3 every time the cabinet door is being closed.The latest pictures are compared to those taken immediately before andthe computer 5 records the changes. In order to cover also the meal casewhen a milk package has been taken out from the refrigerator 1 beforethe meal, and after this withdrawal is considered as non-existent butafter the meal is returned to the refrigerator, the computer 5 comparesthe two latest pictures with the to pictures taken immediately before.The computer program assumes that a certain volume of milk has beenconsumed at this occasion and if a shopping list is ordered from thecomputer, milk will be listed. If the milk package is returned within adetermined time period, for example two hours can be programmed time,the computer records it as the same package. If a longer time haselapsed between the closure of the cabinet when the package was returnedand the closure immediately before, the computer records the milkpackage as a new one. Then the computer records a new“best-before-date”. It happens, of course, that a package is removedfrom the cabinet and then returned unopened. If the package is placedfairly on its initial place within the programmed “approved”out-of-refrigerator time nothing happens. However, if returned to adifferent place the package is marked on the monitor 8 or in the list ina differing colour. This means that the user has to confirm that it isthe same package that previously has been taken from the refrigerator.In opposite case the computer will record it as a new package of thesame article and give it a new “best-before-date”.

Bearing in mind that not all shops get daily deliveries of perishables,such as milk, and in many shops the coordination between deliveries andsales of inter alia milk is imperfect one can not trust a programmedfreshness time but in the computer a shorter time must be tied to thespecific package. As a consequence, for safety reasons the user musthimself be able to record the “best-before-date”, which can take placein two ways. Either the package is placed on the shelf 10 and the camera3 takes pictures of a date stamp on the package which is recorded in thecomputer 5. Alternatively, the user records the date of the stamp whilethe package on the shelf 10 is depicted by the camera 3. In the computer5 time periods have been programmed for various food-stuffs indicatingto which extent the freshness time has been shortened when these are atroom temperature, such as during a meal or when being brought home fromthe shop.

The virtual system of coordinates of the computer 5 is helpful inkeeping a check on specific articles, in particular those which arepacked. When the milk package referred to above is returned to the sameposition from which is was taken no problem occurs. In the computer 5 acertain tolerance value can be programmed for the case that the packagehas been slightly displaced horizontally. Should the package be put intothe refrigerator on a different shelf as compared to the shelf fromwhich it was taken the computer notes that it is an opened package andrecords it together with the oldest “best-before-date” noted for milkpackages in the refrigerator.

If the person loading articles into the refrigerator should put newlybought articles in front of remaining articles of the same kind thesystem of coordinates will help. If the older package remains unmoved orif it has been displaced further into the refrigerator it will bedisplayed on the monitor in a differing colour. Based on thisinformation the user can rearrange the articles or he can take out thepackage with the innermost position next time this kind of article is tobe used.

If articles are loaded in a cabinet in a closely packed arrangement andpictures have been taken by the cameras it may happen that the imageanalyser does not succeed in putting together pictures such that thecomputer 5 recognizes all articles. Even if the mirrors 11 complete thedirect pictures taken by the cameras 3 of the sides of the packagesfacing the cameras and the system of coordinates of the computer 5assigns the mirror images to the right packages the information to theimage analyser could be insufficient for the computer 5 to recognizecertain packages. Now, the unknown articles are shown lighter on themonitor whereby the user is requested to put one article at a time onthe shelf 10 for identification. After, in this way, pictures have beentaken of the articles they can be put back into the cabinet as known bythe computer 5.

FIG. 4 shall illustrate how hidden articles, preferably packages, in arefrigerator are identified by the apparatus. As an assumption, theshelves of the refrigerator are essentially transparent, in practicemeaning shelves made of glass or in bar shape. As a result, suitablyplaced cameras 3 can take pictures of the undersides/bottom surfaces ofthe packages A,B,C,D,E,F. Due to the fact that the position of thecamera 3 which takes the pictures is known, the shelf with the packageconcerned will be identified. Hereby, the bottom pictures are pairedtogether with the correct shelf overviews. The virtual system ofcoordinates serves to give orientation on where on the shelf the articleis placed. By the aid thereof the apparatus then determines whichpackage underside belongs to the unknown article. To the extent that thebottom side of the article is significative the article should now beidentified as the whole bottom side has been depicted except for what ishidden by the bars in case of a bar shelf.

If there is no picture stored in the local computer 5 of the verticalsides of the article a request is sent to the central computer 9 whichshould be able to deliver this information.

The purpose of the pictures of FIG. 4 is to show how the identity of thearticle E is obtained. FIG. 4 a shows schematically a refrigerator 1with open door 2. Adjacent to its upper edge and close to its lockingside a camera 3 is provided. During the closing phase of the door 2 thecamera takes a picture showing the upper part of FIG. 4 b. At the sametime a camera 3′, also disposed in the door 2 vertically below thecamera 3, takes a picture. This picture is converted in the computer 5into a orthogonal projection. As shown in the lower part of FIG. 4 b theconverted picture corresponds well with the picture of the upper part.Accordingly, here a picture is obtained of the bottom side of thepackage designated E which is hidden in the upper picture of FIG. 4 b.If the bottom picture of the package E should be characteristic or if itcan be recognized from the preceding opening the computer 5 or thecentral computer 9 can present a picture of the package/the article E asa whole. In case not an indication on the monitor 8 tells that on theshelf 4 there is an article which has to be placed on the shelf 10 inorder to be registered in the memory of the computer 5.

In the situation shown in FIG. 4 it is, of course, an advantage if amirror 11 is placed behind the shelf 4. Thereby a reflected image can beobtained of one vertical side of the package E. This picture is turnedright in the image analyser 6 and used together with the picture takenby the camera 3′ of the bottom side of the package E for the purpose offinding, for instance in the database of the central computer 9, theremaining sides of the package. Then, on the monitor 8 a picture can bedisplayed according to FIG. 4 c or FIG. 4 d.

The presentation of the content can follow the real arrangement in therefrigerator but it can also show groups of articles according to thedate they were first identified or according to types of articles, e.g.vegetables, meat, beverage, etc. In the later case, by marking in thepicture, information can be given about purchase time and lastconsumption date.

In case a medicine compartment is contained in a refrigerator or if aspecial medicine cabinet is provided it can be supervised by theapparatus. Then, by means of the computer clock, it is possible tocontrol the display of packages of medicines which are to be taken at acertain time during the day so that they are marked in a differingcolour. Moreover, the number of tablets to be administrated at eachoccasion can be shown above the reproduction of the respective packageon the monitor. This method can be used to control that the medicationis followed and, if not, to give a suitable alarm. In the casesdiscussed above the articles can of course also be presented as lists.

The apparatus can be used in connection with a plurality of similarmethods as would be apparent to the man skilled in the art.

1. A method for recording the position and keeping a check on thestorage time of articles stored in a room, a cabinet, a refrigerator, orother space, wherein one or several cameras are positioned in oradjacent to the space to photograph articles having been placed onshelves within the space, said cameras transferring pictures of thearticles to a computer and associated image analyser, which on a monitordisplays pictures put together or lists of the contents of the space,characterized in that after the entry of articles into the space, withthe articles having a contact surface resting against the shelves, thesearticles are identified on pictures taken by the cameras in connectionwith a register programmed into the computer, said articles beingregistered with regard to orientation and position in the space and/orthe time of entry, and that the identification of the articles iseffected via depiction of the contact surface of the articles.
 2. Amethod according to claim 1, characterized in that identification by thecomputer is based upon those articles present in the space at thepreceding registration.
 3. A method according to claim 1, characterizedin that if an article placed in the space cannot be identified by thecomputer this article is displayed on the monitor in an overview picturein a differing colour or lighter than remaining articles.
 4. A methodaccording to claim 1, characterized in that the computer automaticallygives to unidentified articles a unique identification which isregistered together with the pictures.
 5. A method according to claim 1,characterized in that if the computer cannot directly identify anarticle this article is placed on a marked place in the space to bedepicted by at least one camera, the resulting picture or pictures beingthen registered in the computer.
 6. A method according to claim 1,characterized in that if the article is not provided with a bar code itis depicted by one or several cameras of which at least one is directedtowards one or several mirrors fixedly mounted in the space, whereby thesides of the article is registered in the computer.
 7. A methodaccording to claim 1, characterized in that inputting of articles orinformation thereof in the computer takes place by means ofvoice-controlled communication.
 8. A method according to claim 1,characterized in that a virtual system of coordinates has beenprogrammed into the computer making possible for correct back sides ofthe articles to be registered together with front sides of the articlesvisible to the cameras.
 9. A method according to claim 8, characterizedin that in a space, relatively closely packed with various articles sothat the cameras can only transfer picture fragments to the computer, bymeans of the virtual system of coordinates the computer can keep a checkon which articles are represented by the picture fragments.
 10. A methodaccording to claim 1, characterized in that when the freshness time ofan article is about to expire the computer gives a signal on the monitorby showing on an overview picture the article in a framed shape ormarked in a list.
 11. Apparatus for recording the position of articles,said apparatus comprising a space (1) having at least one door (2), oneor several cameras (3) for supervising the interior of the space (1), alocal and/or a central computer (5) to which taken pictures aretransferred by the cameras (3) in order for the pictures to be separatedby an image analyser (6) connected to the computer (5) and the articlespresent on the pictures to be identified for registering in the computer(5), and a monitor (8) connected to the computer and on which picturesof the articles or a list can be displayed on command from acommunication means (7) associated with the computer (5), characterizedin that a virtual system of coordinates is provided in the computer (5)by means of which the orientation and place of each article isdetermined, means for registering the time of entry into the space andwithdrawal from same of each article, and means for depiction of thecontact surface, of the articles, against the shelf.
 12. Apparatusaccording to claim 11, characterized in that in case the computer (5)cannot identify an article entered into the space the computer (5) canbe connected to a central computer (9) which transfers the requestedidentity to the computer (5) to be added into the register thereof. 13.Apparatus according to claim 11, characterized in that if theinformation on an article contained in a picture transferred to theimage analyser (6) from the camera (3) is insufficient for the computer(5) to be able to identify the article, the visible part of the articleis shown in a differing colour in an overview picture presented on themonitor (8).
 14. Apparatus according to claim 11, characterized in thatin case an article is to be entered into the space which is notregistered neither in the computer (5) nor the central computer (9) ashelf (10) of transparent material is provided towards which aredirected at least two cameras (3) which by means of mirrors fixedmounted in the space depict the article from several sides, eventuallyincluding back sides, when the article is placed on the shelf (10). 15.Apparatus according to claim 11, characterized in that a thermalregister for perishables is programmed into the computer (5) accordingto which the remaining “best-before-time” can be registered individuallyand adjusted in dependence on the time the article has been out of thetemperature zone adapted for preserving freshness.
 16. Apparatusaccording to claim 11, characterized in that at least one shelf isprovided with one or several weight sensors.
 17. An apparatus forrecording the position of articles, comprising; a cabinet with aninterior accessed via a door; shelves located in the cabinet interior,the shelves having a supporting surface to hold articles placed withinthe cabinet interior on the supporting surface, the articles having acontact surface which bears against the supporting surface of theshelves; at least one camera located to photograph articles placedwithin the cabinet interior and produce digital images of thephotographed articles, the digital images including a digital image ofthe contact surface of each article bearing against the supportingsurface of the shelves; a computer connected to receive the digitalimages from the camera; an image analyser connected to the computer toanalyze the digital images, including the digital image of the contactsurface, to identify each photographed article for articleidentifications; and an article database storing the articleidentifications, wherein, placement of each of the articles within theinterior is determined by the computer using an analysis of the contactsurface of each article.
 18. The apparatus of claim 17, wherein, thecamera is programmed to photograph a new article placed within thecabinet interior at a set lapsed time after closing the cabinet door,the apparatus further comprising a light which remains illuminatedduring the lapsed time to illuminate the new article, duringphotographing, after the cabinet door has been closed.
 19. The apparatusof claim 17, further comprising: a load sensor mounted to detect achange in a number of articles on one shelf, the sensor being interfacedto the computer to update the article database upon the sensor detectingthe change in the number of articles on the one shelf.
 20. The apparatusof claim 17, wherein, the shelves are translucent to the camera, and thecomputer provides a display of the articles showing the articles asplaced on the shelves.